对于普通话端到端（E2E）自动语音识别（ASR）任务，与基于角色的建模单元相比，基于发音的建模单元可以改善模型培训中的建模单元的共享，但遇到了同音词。在这项研究中，我们建议使用一种新颖的发音意识到的独特字符编码来构建基于E2E RNN-T的普通话ASR系统。所提出的编码是发音基本音节和字符索引（CI）的组合。通过引入CI，RNN-T模型可以在利用发音信息来提取建模单元的同时克服同音问题。通过提出的编码，可以通过一对一的映射将模型输出转换为最终识别结果。我们在Aishell和MagicData数据集上进行了实验，实验结果表明了该方法的有效性。

声学和语言特征是口语识别（LID）任务的重要提示。最近的高级盖系统主要使用缺乏明确语言特征编码的声学特征。在本文中，我们通过将RNN换能器模型集成到语言嵌入框架中，提出了一种基于换能器的新型语言嵌入方法，用于盖子任务。从RNN传感器的语言表示能力的优势中受益，该方法可以利用语音感知的声学特征和盖子任务的明确语言特征。实验是在大规模的多语言Librispeech和Voxlingua107数据集上进行的。实验结果表明，所提出的方法显着提高了盖子任务的性能，分别对内域和跨域数据集的相对改善为12％至59％和16％至24％。

由于使用深度学习模型作为基本功能，语音增强（SE）的性能已大大提高。本文中，我们提出了一种感知对比度拉伸（PC）方法，以进一步提高SE性能。 PC是基于临界频带重要性函数得出的，并应用于修改SE模型的目标。具体而言，目标特征的对比是根据感知重要性拉伸的，从而提高了整体SE性能。与基于后处理的实现相比，将PC纳入培训阶段可以保留性能并减少在线计算。值得注意的是，PC可以与不同的SE模型架构和训练标准结合使用。此外，PC不影响SE模型训练的因果关系或收敛性。 VoiceBank按需数据集的实验结果表明，所提出的方法可以在因果关系（PESQ得分= 3.07）和非causal（PESQ分数= 3.35）SE任务上实现最先进的表现。

本文介绍了一种新的鉴别器受约束的最佳运输网络（DOTN），其对语音增强（SE）执行无监督的域适应，这是语音处理中的重要回归任务。 DOTN旨在通过利用来自源域的知识来估计目标域中噪声语音的清洁参考。据报道，培训和测试数据之间的域移位是在不同领域中学习问题的障碍。虽然丰富的文献存在于对分类的无监督域适应上，但提出的方法，尤其是回归，仍然稀缺，并且通常取决于输入数据的附加信息。所提出的DOTN方法通过生成的对抗性框架来说，从数学分析中统治最佳运输（OT）理论，以帮助评估目标域中的连续标签。在两个SE任务上的实验结果表明，通过延长经典OT制剂，我们提出的DOTN以纯粹无监督的方式优于先前的对抗域适应框架。

In this chapter, we review and discuss the transformation of AI technology in HCI/UX work and assess how AI technology will change how we do the work. We first discuss how AI can be used to enhance the result of user research and design evaluation. We then discuss how AI technology can be used to enhance HCI/UX design. Finally, we discuss how AI-enabled capabilities can improve UX when users interact with computing systems, applications, and services.

An increasing number of public datasets have shown a marked clinical impact on assessing anatomical structures. However, each of the datasets is small, partially labeled, and rarely investigates severe tumor subjects. Moreover, current models are limited to segmenting specific organs/tumors, which can not be extended to novel domains and classes. To tackle these limitations, we introduce embedding learned from Contrastive Language-Image Pre-training (CLIP) to segmentation models, dubbed the CLIP-Driven Universal Model. The Universal Model can better segment 25 organs and 6 types of tumors by exploiting the semantic relationship between abdominal structures. The model is developed from an assembly of 14 datasets with 3,410 CT scans and evaluated on 6,162 external CT scans from 3 datasets. We rank first on the public leaderboard of the Medical Segmentation Decathlon (MSD) and achieve the state-of-the-art results on Beyond The Cranial Vault (BTCV). Compared with dataset-specific models, the Universal Model is computationally more efficient (6x faster), generalizes better to CT scans from varying sites, and shows stronger transfer learning performance on novel tasks. The design of CLIP embedding enables the Universal Model to be easily extended to new classes without catastrophically forgetting the previously learned classes.

Recent advances in self-supervised learning (SSL) in computer vision are primarily comparative, whose goal is to preserve invariant and discriminative semantics in latent representations by comparing siamese image views. However, the preserved high-level semantics do not contain enough local information, which is vital in medical image analysis (e.g., image-based diagnosis and tumor segmentation). To mitigate the locality problem of comparative SSL, we propose to incorporate the task of pixel restoration for explicitly encoding more pixel-level information into high-level semantics. We also address the preservation of scale information, a powerful tool in aiding image understanding but has not drawn much attention in SSL. The resulting framework can be formulated as a multi-task optimization problem on the feature pyramid. Specifically, we conduct multi-scale pixel restoration and siamese feature comparison in the pyramid. In addition, we propose non-skip U-Net to build the feature pyramid and develop sub-crop to replace multi-crop in 3D medical imaging. The proposed unified SSL framework (PCRLv2) surpasses its self-supervised counterparts on various tasks, including brain tumor segmentation (BraTS 2018), chest pathology identification (ChestX-ray, CheXpert), pulmonary nodule detection (LUNA), and abdominal organ segmentation (LiTS), sometimes outperforming them by large margins with limited annotations.

We present Muse, a text-to-image Transformer model that achieves state-of-the-art image generation performance while being significantly more efficient than diffusion or autoregressive models. Muse is trained on a masked modeling task in discrete token space: given the text embedding extracted from a pre-trained large language model (LLM), Muse is trained to predict randomly masked image tokens. Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding. The use of a pre-trained LLM enables fine-grained language understanding, translating to high-fidelity image generation and the understanding of visual concepts such as objects, their spatial relationships, pose, cardinality etc. Our 900M parameter model achieves a new SOTA on CC3M, with an FID score of 6.06. The Muse 3B parameter model achieves an FID of 7.88 on zero-shot COCO evaluation, along with a CLIP score of 0.32. Muse also directly enables a number of image editing applications without the need to fine-tune or invert the model: inpainting, outpainting, and mask-free editing. More results are available at https://muse-model.github.io

Feature selection helps reduce data acquisition costs in ML, but the standard approach is to train models with static feature subsets. Here, we consider the dynamic feature selection (DFS) problem where a model sequentially queries features based on the presently available information. DFS is often addressed with reinforcement learning (RL), but we explore a simpler approach of greedily selecting features based on their conditional mutual information. This method is theoretically appealing but requires oracle access to the data distribution, so we develop a learning approach based on amortized optimization. The proposed method is shown to recover the greedy policy when trained to optimality and outperforms numerous existing feature selection methods in our experiments, thus validating it as a simple but powerful approach for this problem.

Human parsing aims to partition humans in image or video into multiple pixel-level semantic parts. In the last decade, it has gained significantly increased interest in the computer vision community and has been utilized in a broad range of practical applications, from security monitoring, to social media, to visual special effects, just to name a few. Although deep learning-based human parsing solutions have made remarkable achievements, many important concepts, existing challenges, and potential research directions are still confusing. In this survey, we comprehensively review three core sub-tasks: single human parsing, multiple human parsing, and video human parsing, by introducing their respective task settings, background concepts, relevant problems and applications, representative literature, and datasets. We also present quantitative performance comparisons of the reviewed methods on benchmark datasets. Additionally, to promote sustainable development of the community, we put forward a transformer-based human parsing framework, providing a high-performance baseline for follow-up research through universal, concise, and extensible solutions. Finally, we point out a set of under-investigated open issues in this field and suggest new directions for future study. We also provide a regularly updated project page, to continuously track recent developments in this fast-advancing field: https://github.com/soeaver/awesome-human-parsing.